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TO THE HJCATIONAL RESOURCES 
INFOPV .riON CENTER (ERIC)." 



INTRODUCTION 



Computer-mediated communication (CMC) has frequently been used in 
establishing a discussion forum for academics. There are many experiences from 
its use especially in connection with different higher education courses (Hansen 
& Gladfelter, 1995; McComb 1994; Steeples, Goodyear, & Mellar, 1994), but 
CMC has also been used for providing a communication forum for teachers, 
researchers and administrative staff (Kaye, Mason, & Harasim, 1989). The 
central reason for this interactive use of CMC is that it facilitates and increases 
interaction and collaboration among participants (Wells, 1993). There are also 
many research results (Burge, 1994; Mason, 1993) on positive experiences and 
learning effects gained through collaborative CMC discussions. 

In terms of higher education pedagogy, CMC has at least two relevant 
features. In the first place, learning in a CMC environment is largely dependent 
on students' own activity and motivation and, thus, CMC is seen to promote 
self-direction among students (Seaton, 1993). In the same vein, McComb (1994) 
emphasizes that CMC learners must take responsibility for their own studies 
and that they should be seen as active creators of learning. In this respect, CMC 
promotes an essential fimction of higher education studies which, according to 
Gow and Kember (1990) is to educate independent learners. In the second place, 
CMC can be regarded to include such characteristics that are. beneficial in 
practising argumentative writing and argumentation and critical thinking skills 
whose promotion is, along with self-direction, a central goal of higher education 
(e.g., Banta, 1993). First, the time and place independent as well as 
as 5 mchronous nature of CMC provides the students with a flexible environment 
in which they can consider and construct their ideas and arguments carefully; 
second, the written form of interaction necessitates explicit expression of 
thoughts; and third, in a learning environment based on many-to-many 
communication students can make use of various approaches and points of 
view to matters under examination (Harasim, 1990; Steinberg, 1992). 

Mature argumentation and critical thinking skills are needed e.g. in 
academic discussions and debates when one has to support one's statements 
with relevant grounds. Voss and Means (1991) characterize persons possessing 
mature argumentation skills as people who are able to shape and select such 
grounds that support stated claims. In addition, persons skilled in 
argumentation ground their positions from many points of view. 

The importance of teaching argumentation and critical thinking skills in 
higher education has been widely emphasized (e.g., Atwater, 1991; Terenzini, 
Spinger, Pascarella, & Nora, 1995). Experiences from teaching these skills have 
shown that students benefit from such learning environments in which they are 
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actively engaged in debate and constructive dialogue (Colbert, 1987; Littlefield, 
1995). In addition, previous CMC literature (Seaton, 1993; Steinberg, 1992) 
suggests that CMC is suited to producing written argumentation and dialogue. 
In this str.dy academic argumentation and critical thinking were practised in a 
learning environment based on the use of e-mail. 



RESEARCH ON CMC 

Previous CMC research has frequently been focused on various survey 
questionnaires and user interviews as well as on many practical experiments 
and case studies, but less frequently on the evaluation of the content of CMC 
messages (see Mason, 1992). However, the need for the content analysis method 
is commonly recognized (Kuehn, 1994; Mason, 1992) since, as Henri (1992) puts 
it, content analysis provides a possibility to reach a better understanding of 
learning in a CMC enviromnent since it clarifies the students' cognitive 
processes and ways of handling information during studies. 

In those studies that have concentrated on CMC messages the focus has 
been on the structure and intent (Hailes, 1986; Kuehn, 1993; Smeltzer, 1992) and 
the thematic tent (Friedman & McCullough, 1992) of the messages as well as 
on the differences between electronic and face-to-face discourse (Black, Levin, 
Mehan, & Quinn, 1983). In addition, in a recent study by Newman, Webb and 
Cochrane (1995) the indicators of critical vhinking in e-mail messages were 
analyzed. However, it appears that the content of e-mail messages in terms of 
argumentation has not been studied earlier. Although there are earlier 
experiences from CMC applications in which e-mail has been used as a forum 
for argumentation and debate (Charlton 1993; Pugh, 1993), the literature lacks 
of systematic analyses on the use of CMC for this purpose. This kind of analysis 
would produce knowledge on the relevance of CMC for practising 
argumentation that is needed in developing study methods which could meet 
especially the goals of higher education. This study seeks to increase knowledge 
of CMC in this respect. 

The research tasks were a) to examine the level and development of 
argumentation and counterargumentation in the students' e-mail messages, and 
b) to clarify associations between the quality and quantity of argumentation and 
counterargumentation and two different e-mail study modes. 







3 



METHOD 

Subjects 

The e-mail study period was implemented at the Department of Education, in 
the University of Jyvaskyla, Finland, during the fall term 1990. In the beginning, 
32 students volunteered to complete the M.Ed level introductory course in the 
sociology of education through six week e-mail discussions. Since one student 
dropped out, 31 completed the studies. Two top students in the field of educa- 
tion near their graduation were also recruited as tutors of the e-mail studies. 
Other 193 students of the course engaged in traditional self-study of the set 
books of the course. 

A majority of the e-mail students were female (58%) and 23 years of age 
or younger (52%). In addition, most of them were students of humanities (45%) 
and education (15%) at the early stages of their studies. 

E-mail Study Organization 

During the e-mail studies the students were supposed to write at least two 
messages a week related to the set books (Broady, 1986; Takala, 1989) and the 
course lectures in order to pass the course. 

Four individual e-mail study groups , /ere established by the researcher so 
that there were both male and female students in each group. Two of the 
groups engaged in the seminar mode and the other two groups in the 
discussion mode of e-mail study. In the seminar mode the tutor selected the 
discussion topics from the course contents and gave the students regular 
content-related feedback. The tutor's role was to act as a teacher and a leader. 
The discussion mode, by contrast, was based on the students' self-direction: they 
had the freedom to select the discussion themes from the study contents by 
themselves, and thus, they could write their texts on those topics they found 
interesting and worth discussing. The tutor acted, rather, as a co-worker and a 
facilitator of learning and gave feedback to the students only occasionally. 

The tutors tutored two groups simultaneously and worked with the same 
book during the whole e-mail study period. During the first half of the studies 
the seminar groups studied Takala's book and the discussion groups Broady's 
book, and during the second half the order was reversed. 

The configuration used was an electronic mail (Elm) for Unix. The students 
could use the terminals located on campus at any time convenient for them. All 
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the messages the students sent were delivered to each member within each 
individual study group via the mailing list attached to the mailing system. 
Thus, the configuration provided the students with a forum for many-to-many 
communication (see Harasim, 1990). The students could also scan and reread 
the messages whenever they wanted. 

The participants did not know each other before the experiment and they 
also could use pen names when signing their messages. No face-to-face 
meetings were organized during the study. The students met each other only 
once in a single face-to-face session held before the studies when they were 
taught how to use the e-mail program. 

The aim of the e-mail discussions was to practise informal argumentation. 
A literature review on argumentation was delivered to the participants before 
the Studies in order to get them familiar with the concept and procedural struc- 
ture of argumentation. The students practised argumentation by presenting in 
their e-mail texts their own grounded standpoints, critique and 
counterarguments as well as defended themselves when they themselves 
received critique. In order to produce argumentative interaction the tutors 
encouraged the students to evoke and engage in debates on the issues dealt 
with. The tutors also gave the students guidance in argumentatiori by including 
good argumentation in their own messages and by pointing out as "model 
messages" those students' texts which included solid argumentation. 



Data of the Study 

The total number of the messages sent by the students during the study was 
489. From these the messages relevant in terms of the study of the course 
contents (n = 441) were included in the data. The 48 excluded messages dealt 
solely with technical matters and practical questions related to the e-mail 
studies. 



Data Analyses 

The content analysis of the data focused on the manifestation of argumentation 
in the messages. Generally argumentation refers to grounding of stated claims 
(Toulmin, Rieke, & Janik, 1984). Yet, when a claim that is grounded is a counter- 
claim in nature targeted against someone else's opinion argumentation is called 
counterargumentation. According to Lo Cascio (1995), counterargumentation 
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usually concerns an attack against the validity or relevance of some previous 
argumentation. Since the students in this study were especially encouraged to 
^rigage in counterargumentation by presenting critique and attacking each 
others' arguments, coimterargumentation was of special interest also in the 
analyses. 

The data analysis of the stt dy proceeded through two stages. In the first 
stage, the general level of argumentation of all the messages was analyzed 
focusing on both counterargumentation and argumentation that did not present 
contradicting standpoints to what other students had said. The second stage of 
the analysis concentrated on counterargumentation only. 

Argumentation Analysis 

The argumentation analysis of the data was carried out by using an e-mail 
message as a unit of analysis. The messages were classified into three categories 
on the basis of the researcher's general estimation of the messages' Level of 
argumentation' (variable XI): good (value 2), moderate (value 1), and poor 
(value 0). 

The messages rated to include good argumentation consisted mainly of 
text in which the writer had presented standpoints with supporting reasons (cf. 
Colder & Coirier, 1994; Hintikka & Bachman, 1991) which, at least in places, 
provided the claim with a relevant and wide scope of support (see Perkins, 
1986; Voss & Means, 1991). In the messages rated to include moderate 
argumentation, the proportion of argumentative text was not as large as in the 
messages classified into the highest category. Thus, the moderate messages 
could also include relatively much non-argumentative text in which the writer 
had not presented positions but, rather, facts relating to various states of affairs. 
Finally, in the messages rated to consist of poor argumentation the writer had 
either not put forward a position at all or had presented standpoints without 
any supporting grounds (cf. Colder & Coirier, 1994). The texts classified into 
this poorest category resembled often summaries in which the writer had 
presented various states of affairs but not expressed any opinions. 

The reliability of the analysis was estimated by having another person 
classify 44 messages (10%) independently and by calculating the inter-coder 
reliability coefficient (see Bryman & Cramer, 1990, pp. 71-72). The reliability (r) 
was .71 {p < .01). 
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Analysis of Counterargumentation 

First in the analysis of counterargumentation a variable 
'Counterargumentativeness' (X3) was formed on the basis whether a message 
included counter-claims directed against fellow students' or tutors' opinions or 
opinions encountered in the study contents. The messages which included one 
or more counter-claims were named as 'Counterargumentative messages' (value 

1) and those that did not as 'Non-counterargumentative messages' (value 0). 

The further analysis concerned the counterargumentative messages only (N 
= 55). All the counterarguments (a counter-claim and its possible groundings) in 
the messages were graded into three classes according to the level of the 
grounding of the counter-claim: good (value 2), moderate (value 1), and poor 
(value 0). In the counterarguments classified into the highest category (good) the 
counter-claim was supported by relevant and sufficient reasons; in those 
classified into the middle category (moderate), the grounding was defective 
although somehow supportive of the counter-claim; and, in those classified into 
the lowest category (poor), either the grounds were totally irrelevant or the 
counter-claim was not supported at all. 

All the messages were then classified into three categories on the basis of 
the 'Level of counterargumentation' (variable X2) included in them: good (value 

2) , moderate (value 1), and poor (value 0). Each message's category was based 
on the mean value of the counterarguments found in it: values from 0 to 0.5 = 
0, values from 0.51 to 1.50 = 1, and values from 1.51 to 2.0 = 2. 

Statistical Analyses 

Log-linear models were used in analyzing the multidimensional associations of 
the categorical variables of the study (see Fienberg, 1981). Two log-linear 
analyses according to a symmetrical mode of inquiry (see Kennedy, 1988) were 
carried out. In the first analysis associations between variables 'Level of 
argumentation' (XI: good, moderate or poor), 'Counterargumentativeness of 
message' (X3: yes vs. no), 'Time of sending the message' (X4: during the first vs. 
second half of e-mail study), and 'Mode of the e-mail study' (X5: seminar vs. 
discussion) were examined. The second analysis concerned the 
counterargumentative messages only and examined associations between 
variables 'Level of counterargumentation' (X2: good, moderate or poor), 'Time 
of sending the message', and 'Mode of the e-mail study'. 
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The analyses were implemented by starting from a saturated model in 
which all the possible main and interaction effects of the variables were 
included. After this all those parameters that were not statistically significant 
were dropped from the model step by step by starting from higher-order terms, 
ending at a minimal acceptable model which fitted the data and contained as 
few statistically significant parameters as possible (see Andersen, 1980). The 
analyses are described in Tables 1 and 2. 



TABLE 1 

Log-linear Analysis 1 (n = 441) 



Parameter 


Level of Xl^ 


Estim. 


SE 


Z 


P 


Saturated Model 

XI (Level of argumentation) 


(0) 


.27 


.12 


2.21 


♦ 




(1) 


.34 


.11 


3.16 




X3 (Counterargumentativeness of message) 


.92 


.08 


10.61 


♦♦♦ 


X4 (Time of sending the message) 




-.06 


.09 


-.66 


ns. 


X5 (Mode of the e-mail study) 




-.15 


.09 


-1.68 


ns. 


X1.X3 


(0) 


.42 


.12 


3.49 


♦♦♦ 




(1) 


-.11 


.11 


-1.06 


ns. 


X1.X4 


(0) 


.29 


.12 


2.37 


♦ 




(1) 


-.16 


.11 


-1.46 


ns. 


X1.X5 


(0) 


.19 


.12 


1.56 


ns. 




(1) 


.07 


.11 


.67' 


ns. 


X3.X4 




-.09 


.09 


-.99 


ns. 


X3.X5 




.25 


.09 


2.92 


♦♦ 


X4.X5 




.07 


.09 


.85 


ns. 


X1.X3.X4 


(0) 


-.11 


.12 


-.92 


ns. 




(1) 


.05 


.11 


.50 • 


ns. 


X1.X3.X5 


(0) 


-.09 


.12 


-.73 


ns. 




(1) 


.04 


.11 


.38 


ns. 


X1.X4.X5 


(0) 


.05 


.12 


.44 


ns. 




(1) 


.09 


.11 


.85 


ns. 


X3.X4.X5 




-.14 


.09 


-1.64 


ns. 


X1.X3.X4.X5 


(0) 


-.04 


.12 


-.29 


ns. 




(1) 


.12 


.11 


1.15 


ns. 


Minimal Acceptable Model (G^ = 18.60, df = 
XI 


13, p = .14) 
(0) 


.25 


.11 


2.20 


♦ 


X3 


(1) 


.33 


.10 


3.33 






.93 


.08 


11.33 




X4 




-.13 


.06 


-2.29 


♦ 


X5 




-.11 


.08 


-1.48 


ns. 
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TABLE 1 (continues) 



Parameter 


Level of Xl” 


Estim, 


SE 


Z 


P 


X1.X3 


(0) 


.45 


.11 


4.05 


4-4-4- 




(1) 


-.10 


.10 


-.99 


ns. 


X1.X4 


(0) 


.18 


.07 


2.61 


♦♦ 




(1) 


-.09 


.07 


-1.28 


ns. 


X3.X5 




.29 


.08 


3.71 


4-4-4- 


■’Since variable XI has three classess there are two estimated parameters in the analysis 
* p < .05; ** p < .01.; *** p < .001. 

TABLE 2 

Log-linear Analysis 2 (n = 55) 




Parameter 


Level of X2' 


Estim, 


SE 


Z 


P 


Saturated Model 


X2 (Level of counterargumentation) 


(1) 


.04 


.21 


.17 


ns. 




(2) 


.32 


.21 


1.54 


ns. 


X4 (Time of sending the message) 




-.02 


.16 


-.13 


ns. 


X5 (Mode of the e-mail study) 




-.37 


.16 


-2.31 


Ji- 


X2.X4 


(1) 


.21 


.21 


.99 


ns. 




(2) 


-.01 


.21 


-.05 


ns. 


X2.X5 


(1) 


.56 


.21 


2.60 


** 


X4.X5 


(2) 


-.28 


.21 


-1.33 


ns. 




.20 


.16 


1.29 


ns. 


X2.X4.X5 


(1) 


-.01 


.21 


-.06 


ns. 


Minimal Acceptable Model (G^ = 5.41, df = 


(2) 

= 6, p = .49) 


-.00 


.21 


-.02 


ns. 


X2 


(1) 


.07 


.23 


.32 


ns. 


X5 


(2) 


.35 


.23 


1.56 


ns. 




-.43 


.18 


-2.50 


4- 


X2.X5 


(1) 


.69 


.23 


3.01 


** 


ao: • 1_1_ ■v/'r'* 1 1 


(2) 


-.28 


.23 


-1.23 


ns. 



''Since variable X2 has three classess there are two estimated parameters in the analysis 
* p < .05; ** p< .01. 



The analyses revealed that there were statistically significant associations 
between four individual pairs of variables. The first analysis (Table 1) indicated 
that variable 'Level of argumentation' was associated both with variables 
'Counterargumentativeness of message' and 'Time of sending the message', and 
that 'Counterargumentativeness of message' was also associated with variable 
'Mode of the e-mail study'. Ihe second analysis (Table 2) indicated an 
association between variables 'Level of counterargumentation' and 'Mode of the 
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stu ly . Furth6rmor6/ both snslysos indicstsd thst no statisticslly 
significant higher-order interaction effects among the variables were found. 



RESULTS 

Argumentation 



TABLE 3 

Level of Argumentation of the Messages 



Level of 
argumentation 


Counterargumentativeness 
of the message 


Time of sending the 
message 






yes 


no 


1. half 


2. half 


Total 




f(%) 


f(%) 


f(%) 


f(%) 


f(%) 


2 (Good) 


14(26) 


42(11) 


22(11) 


34(14) 


56(13) 


1 (Moderate) 


27(49) 


135(35) 


63(31) 


99(41) 


162(37) 


0 (Poor) 


14(25) 


209(54) 


117(58) 


106(45) 


223(50) 


Total 


55(100) 


386(100) 


202(100) 


239(100) 


441(100) 


Mean 


1.00 


.57 


.53 


.70 


.62 


SD 


.72 


.68 


.69 


.71 


.70 



The results in Table 3 indicate that the general level of argumentation in the 
students' messages was rated as quite poor: in a half of the messages the level 
was rated as poor, in about every third (37%) as moderate, and only in about 
every tenth message (13%) as good. 

Table 3 also represents the nature of the statistically significant associations 
of the level of argumentation (XI) with counterargumentativeness of the 
messages (X3) and time of sending the message (X4) illustrated in Table 1. Table 
3 indicates, first, that the level of argumentation was better in 
counterargumentative messages (M = 1.00) compared to the non- 

counterargumentative ones (M = .57), and that this finding resulted mainly from 
the contrast between the amount of messages rated as poor (25% vs. 54%). 
Second, the results indicate that the level of argumentation improved during the 
e-mail study period: the level was higher among the messages sent during the 
latter half of e-mail study (M = .70) compared to those sent during the first half 
(M = .53). The difference resulted again mainly from the contrast between the 
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messages rated as poor (45% vs. 58%). In addition, the result that the third 
order association between the level of argumentation (XI), time of sending th^ 
message (X4) and the mode of the e-mail study (X5) was not statistically 
significant (see Table 1) means that the improvement in the level of 
argumentation was not dependent on the mode of the e-mail study. 

Counterargumentation 

TABLE 4 

Counterargumentation in the Messages 



Variable 


Mode of the e-mail study 






Seminar 


Discussion 


Total 




f(%) 


f(%) 


/(%) 


Counterargumentativeness (N = 441) 








Yes 


17(7) 


38(19) 


55(13) 


No 


226(93) 


160(81) 


386(87) 


Total 


243(100) 


198(100) 


441(100) 


Level of counterargumentation (N = 55) 








2 (Good) 


2(12) 


11(29) 


13(24) 


1 (Moderate) 


5(29) 


21(55) 


26(47) 


0 (Poor) 


10(59) 


6(16) 


16(29) 


Total 


17(100) 


38(100) 


55(100) 


Mean 


.53 


1.13 


.95 


SD 


.72 


.67 


.73 



Table 4 indicates that counterargumentation in the students' messages was quite 
rare since only 13% of the messages included counterargumentation. 
Nevertheless, in those 55 messages in which counterargumentation took place 
its level was rated as quite good (M = .95): in about a quarter (24%) of the 
messages the level was rated as good, in about a half (47%) as moderate and 
only in about every third message (29%) as poor. 

In addition, the nature of the associations of both the 
counterargumentativeness (X3, Table 1) and the level of counterargumentation 
of the messages (X2, Table 2) with the mode of the e-mail study (X5) are also 
represented in Table 4. The results indicate, first, that the students engaged in 
the discussion mode of e-mail study produced more (19% vs. 7%) 
counterargumentation than the students engaged in the seminar mode, and 
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second, that the level of counterargumentation was also higher among the 
discussion mode. The difference between the means was 1.13 vs. .72. A 
noteworthy result (see Table 2) was also that although the level of 
counterargumentation among the counterargumentative messages slightly 
improved during the e-mail study period the development was not statistically 
significant. 



DISCUSSION 

The results indicated that the level of argumentation in the students' messages 
improved during the e-mail study period; during the last half of the studies the 
students wrote more argumentative texts than during the first half. This result 
might be construed to be a consequence of the development in the students' 
skills to write argumentative texts. However, this inference is contradictory with 
a large body of previous evidence (see Terenzini et al., 1995; McMillan, 1987), 
which suggests that to achieve improvement in students' argumentation and 
critical thinking skills needs a longer time and that through short-termed 
instructional arrangements it is difficult to produce any measurable impacts. 
Thus, it is important to note that the reason for the improvement of the level of 
argumentation may also be the fact that after the students had got through their 
first difficulties with the new study method they internalized the idea of the 
studies and also got familiar with each other and, thus, during the later stages 
of the course felt more encouraged to express themselves and present their own 
points of views and arguments. At all events, the results are promising and 
e-mail is a reasonable tool in practising argumentation and 
warrants further investigation of the topic. This conclusion is supported also by 
earlier studies reporting on benefits of CMC in practising argumentation and 
critical thinking (Ahern, Peck, & Laycock, 1992; Marthmen, 1992, 1995; Newman 
et al., 1995). However, it should be noted that the subjects of the experiment 
were volunteers who may have beeri more experienced and interested in 
studying with computers than the average student. Hence, the use of randomly 
sampled subjects might have led to less positive results (see Hiltz, 1990). 

The results indicated also that the level of argumentation was higher in the 
counterargumentative messages compared to those in which the writer had not 
attacked other peoples' standpoints. In addition, although the total number of 
counterargumentative messages proved fairly low (13%), counterargumentation 
was distinctly more common and also at a higher level in the discussion groups. 
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in which the students were given freedom to be self -directive, compared to the 
seminar groups in which the studies proceeded, rather, under the tutor's 
control. These results suggest, first, that interaction including critique and 
debates between students should be encouraged when aiming at producing 
argumentative writing and practising argumentation. Second, the results suggest 
that an e-mail environment in which the students are given freedom to be self- 
directive and the tutor acts a facilitator of studies is more favourable in 
producing debates and critical interaction between students compared to e-mail 
studies in which the role of the tutor is more dominant. The conclusions are 
parallel with previous studies reporting on benefits of academic debates and 
disputes in teaching argumentation and critical thinking (Colbert, 1987; 
Littlefield, 1995) as well as by other current results (Ahern et al., 1992; 
Marttunen, 1992) emphasizing the benefits of conversational and free-form style 
of CMC in producing argumentative discussions. 

A noteworthy result in this study was that the average level of 
argumentation in the students' messages was considered to be quite poor. This 
may, at least partially, result from the fact that the students were in the early 
phases of their undergraduate studies, and thus, their argumentation skills 
probably had not been fully developed yet. This supposition is supported by 
previous findings indicating weak argumentation skills among Finnish students, 
especially in the beginning of their higher education studies (Martttmen, 1994). 
Furthermore, according to Pascarella (1989), just participating in university's 
intellectual and critical learning environment promotes the students' 
argumentation skills. Hence, if the participants in this experiment were more 
experienced the messages' level of argumentation could have been expected to 
be better. 

The small proportion of counterargumentative messages indicated also that 
disputes and debates between the students were quite uncommon even if the 
students were encouraged to exchange opinions and critique each other's texts. 
This finding, along with the result showing poor argumentation, may result 
from the Finnish study culture, which at upper secondary as well as at 
undergraduate level of education does not encourage students to debate and 
exchange opinions. Instead, the current teaching methods are still, to a large 
extent, based on transmission of knowledge from the teacher to the students at 
the upper secondary level and on lectures and self-study of set books at the 
university level. Hence, the students in this study may not have been familiar 
with the study method based on critical discussions, which also may have 
contributed to the relatively poor level of argumentation in their texts. 
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These results point to the need to develop such study methods and learning 
environments in the Finnish study system, in which debates, critical interaction 
and exchanging of opinions between students is promoted and encouraged. 
Furthermore, the study suggests that e-mail discussions are a reasonable way to 
establish such environments, and thus, also a feasible working method when 
aiming at reaching one fundamental objective of higher education: to teach the 
students a critical attitude towards knowledge. 
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